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Abstract. We analyze the problem of computing a correlated equilibrium that optimizes some objective (e.g., social 
welfare). Papadimitriou and Roughgarden [2008] gave a sufficient condition for the tractability of this problem; 
however, this condition only applies to a subset of existing representations. We propose a different algorithmic 
approach for the optimal CE problem that applies to all compact representations, and give a sufficient condition 
that generalizes that of Papadimitriou and Roughgarden [2008]. In particular, we reduce the optimal CE problem to 
the deviation-adjusted social welfare problem, a combinatorial optimization problem closely related to the optimal 
social welfare problem. This framework allows us to identify new classes of games for which the optimal CE 
problem is tractable; we show that graphical polymatrix games on tree graphs are one example. We also study the 
problem of computing the optimal coarse correlated equilibrium, a solution concept closely related to CE. Using a 
similar approach we derive a sufficient condition for this problem, and use it to prove that the problem is tractable 
for singleton congestion games. 

1 Introduction 

A fundamental class of computational problems in game theory is the computation of solution concepts of finite games. 
Much recent effort in the literature has concerned the problem of computing a sample Nash equilibrium [Chen & Deng, 
2006; Daskalakis et ai, 2006; Daskalakis & Papadimitriou, 2005; Goldberg & Papadimitriou, 2006]. First proposed 
by Aumann [1974; 1987], correlated equilibrium (CE) is another important solution concept. Whereas in a mixed 
strategy Nash equilibrium players randomize independently, in a coiTelated equilibrium the players can coordinate 
their behavior based on signals from an intermediary. 

Correlated equilibria of a game can be formulated as probability distributions over pure strategy profiles satisfying 
certain linear constraints. The resulting linear feasibiUty program has size polynomial in the size of the normal form 
representation of the game. However, the size of the normal form representation grows exponentially in the number 
of players. This is problematic when games involve large numbers of players. Fortunately, most large games of prac- 
tical interest have highly-structured payoff functions, and thus it is possible to represent them compactly. A line of 
research thus exists to look for compact game representations that are able to succinctly describe structured games, 
including work on graphical games [Kearns et ai, 2001] and action-graph games [Bhat & Leyton-Brown, 2004; Jiang 
et ai, 2011]. But now the size of the linear feasibility program for CE can be exponential in the size of compact 
representation; furthermore a CE can require exponential space to specify. 

The problem of computing a sample CE was recently shown to be in polynomial time for most existing compact 
representations [Papadimitriou & Roughgarden, 2008; Jiang & Leyton-Brown, 2011]. However, since in general there 
can be an infinite number of CE in a game, finding an arbitrary one is of limited value. Instead, here we focus on 
the problem of computing a coiTelated equilibrium that optimizes some objective. In particular we consider two kinds 
of objectives: (1) A linear function of players' expected utilities. For example, computing the best (or worst) social 
welfare coiTesponds to maximizing (or minimizing) the sum of players' utilities, respectively. (2) Max-min welfare: 
maximizing the utility of the worst-off player. (More generally, maximizing the minimum of a set of linear functions of 
players' expected utilities.) We are also interested in computing optimal coarse correlated equilibrium (CCE) [Hannan, 
1957]. It is known that the empirical distribution of any no-external-regret learning dynamic converges to the set of 
CCE, while the empirical distribution of no-internal-regret learning dynamics converges to the set of CE (see e.g. 
[Nisan et ai, 2007]). Thus, optimal CE / CCE provide useful bounds on the social welfare of the empirical distributions 
of these dynamics. Optimal CE / CCE can also be used as bounds on optimal NE since CE and CCE are both relaxations 
of NE. Hence they are also useful for computing (bounds on) the price of anarchy and price of stability of a game. 



We are particularly interested in the relationship between the optimal CE / CCE problems and the problem of 
computing the optimal social welfare outcome (i.e. strategy profile) of the game, which is exactly the optimal social 
welfare CE problem without the incentive constraints. This is an instance of a line of questions that has received much 
interest from the algorithmic game theory community: "How does adding incentive constraints to an optimization 
problem affect its complexity?" This question in the mechanism design setting is perhaps one of the central questions 
of algorithmic mechanism design [Nisan & Ronen, 2001]. Of course, a more constrained problem can in general 
be computationally easier than the relaxed version of the problem. Nevertheless, results from complexity of Nash 
equiUbria and algorithmic mechanism design suggest that adding incentive constraints to a problem is unlikely to 
decrease its computational difficulty. That is, when the optimal social welfare problem is hard, we tend also to expect 
that the optimal CE problem will be hard as well. On the other hand, we are interested in the other direction: when 
it is the case for a class of games that the optimal social welfare problem can be efficiently computed, can the same 
structure be exploited to efficiently compute the optimal CE? 

The seminal work on the computation of optimal CE is [Papadimitriou & Roughgarden, 2008]. This paper consid- 
ered the optimal linear objective CE problem and proved that the problem is NP-hard for many representations, while 
tractable for a couple of representations. In particular, the representations shown to be NP-hard include graphical 
games, polymatrix games, and congestion games. These hardness results, although nontrivial, are not surprising: the 
optimal social welfare problem is already NP-hard for these representations. On the tractabiUty side, Papadimitriou 
and Roughgarden [2008] focused on so-called "reduced form" representations, meaning representations for which 
there exist player-specific partitions of the strategy profile space into payoff-equivalent outcomes. They showed that if 
a particular separation problem is polynomial-time solvable, the optimal CE problem is polynomial-time solvable as 
well. Finally, they showed that this separation problem is polynomial-time solvable for bounded-treewidth graphical 
games, symmetric games and anonymous games. 

Perhaps most surprising and interesting is the form of Papadimitriou and Roughgarden's sufficient condition for 
tractability: their separation problem for an instance of a reduced-form-based representation is essentially equivalent 
to solving the optimal social welfare problem for an instance of that representation with the same reduced form but 
possibly different payoffs. In other words, if we have a polynomial-time algorithm for the optimal social welfare prob- 
lem for a reduced-form-based representation, we can turn that into a polynomial-time algorithm for the optimal social 
welfare CE problem. However, Papadimitriou and Roughgarden's sufficient condition for tractability only applies to 
reduced-form-based representations. Their definition of reduced forms is unable to handle representations that exploit 
linearity of utility, and in which the structure of player p's utility function may depend on the action she chose. As a 
result, many representations do not fall into this characterization, such as polymatrix games, congestion games, and 
action-graph games. Although the optimal CE problems for these representations are NP-hard in general, we are inter- 
ested in identifying tractable subclasses of games, and a sufficient condition that applies to all representations would 
be helpful. 

In this article, we propose a different algorithmic approach for the optimal CE problem that applies to all com- 
pact representations. By applying the ellipsoid method to the dual of the LP for optimal CE, we show that the 
polynomial-time solvability of what we call the deviation-adjusted social welfare problem is a sufficient condition 
for the tractability of the optimal CE problem. We also give a sufficient condition for tractability of the optimal CCE 
problem: the polynomial-time solvability of the coarse deviation-adjusted social welfare problem, which we show 
reduces to the deviation-adjusted social welfare problem. We show that for reduced-form-based representations, the 
deviation-adjusted social welfare problem can be reduced to the separation problem of Papadimitriou and Roughgar- 
den [2008]. Thus the class of reduced forms for which our problem is polynomial-time solvable contains the class 
for which the separation problem is polynomial-time solvable. More generally, we show that if a representation can 
be characterized by "linear reduced forms", i.e. player- specific linear functions over partitions, then for that repre- 
sentation, the deviation-adjusted social welfare problem can be reduced to the optimal social welfare problem. As an 
example, we show that for graphical polymatrix games on trees, optimal CE can be computed in polynomial time. 
Such games are not captured by the reduced-form framework.' The key feature of these representations upon which 
our argument relies is that the partitions for player p (which characterize the structure of the utility function for p) do 
not depend on the action chosen by p. 

On the other hand, representations like action-graph games and congestion games have action-specific structure, 
and as a result the deviation-adjusted social welfare problems and coarse deviation-adjusted social welfare problems on 

' In a recent paper Kamisetty et al. [20 11] has independently proposed an algorithm for optimal CE in graphical polymatrix games 
on trees. They used a different approach that is specific to graphical games and graphical polymatrix games, and it is not obvious 
whether their approach can be extended to other classes of games. 



these representations are structured differently from the corresponding optimal social welfare problems. Nevertheless, 
we are able to show a polynomial-time algorithm for the optimal CCE problem on singleton congestion games [leong 
et al., 2005], a subclass of congestion games. We use a symmetrization argument to reduce the optimal CCE prob- 
lem to the coarse deviation-adjusted social welfare problem with player-symmetric deviations, which can be solved 
using a dynamic-programming algorithm. This is an example where the optimal CCE problem is tractable while the 
complexity of the optimal CE problem is not yet known. 

2 Problem Formulation 

Consider a simultaneous-move game G = {Af, {Sp}p^j\f, {w^jpeA/'), where J\f = {1, ... ,n} is the set of players. 
Denote a player p, and player p's set of pure strategies (i.e., actions) Sp. Let m = maxp \Sp\. Denote a pure strategy 
profile s = (si, . . . , Sn) & S, with Sp being player p's pure strategy. Denote by S-p the set of partial pure strategy 
profiles of the players other than p. Let be the vector of player p's utilities for each pure profile, denoting player 
p's utihty under pure strategy profile s as u^. Let w be the vector of social welfare for each pure profile, that is 
w = ^p(zj^ uP, with Ws denoting the social welfare for pure profile s. 

Throughout the paper we assume that the game is given in a representation with polynomial type [Papadimitriou, 
2005; Papadimitriou & Roughgarden, 2008], i.e., that the number of players and the number of actions for each player 
are bounded by polynomials of the size of the representation. 

2.1 Correlated Equilibrium 

A correlated distribution is a probability distribution over pure strategy profiles, represented by a vector x G R^, 
where M = \Sp\. Then Xg is the probability of pure strategy profile s under the distribution x. 

Definition 1. A correlated distribution x is a correlated equilibrium f CE) if it satisfies the following incentive con- 
straints.- for each player p and each pair of her actions Sp, 

E i<-,-^%-Ms-.>o, (1) 

s — S — p 

where the subscript "is-p " ( respectively "js^p ") denotes the pure strategy profile in which player p plays i ( respec- 
tively j) and the other players play according to the partial profile S-p G S-p. 

Intuitively, when a trusted intermediary draws a strategy profile s from this distribution, privately announcing to each 
player p her own component Sp, p will have no incentive to choose another strategy, assuming others follow the 
suggestions. We write these incentive constraints in matrix form as C/x > 0. Thus [/ is an A'' x M matrix, where 
N = J2p \ Sp\'^. The rows of U, corresponding to the left-hand sides of the constraints (1), are indexed by {p, 
where p is a player and i,j e Sp are a pair of p's actions. Denote by Ug the column of U corresponding to pure 
strategy profile s. These incentive constraints, together with the constraints 

a;>0, = 1, (2) 

ses 

which ensure that a; is a probability distribution, form a linear feasibility program that defines the set of CE. The 
problem of computing a maximum social welfare CE can be formulated as the LP 

max w^x (P) 



Ux>0, x>0,^Xs = l 



Another objective of interest is the max-min welfare CE problem: computing a CE that maximizes the utility of 
the worst-off player. 

max r (3) 

XsU^>r \/p (4) 



E 



Ux>0, x>0,^Xs = l 

s&S 



Another solution concept of interest is coarse correlated equilibrium (CCE). Whereas CE requires that each player 
has no profitable deviation even if she takes into account the signal she receives from the intermediary, CCE only 
requires that each player has no profitable unconditional deviation. 

Deiuiition 2. A correlated distribution x is a coarse correlated equilibrium ( CCE) if it satisfies the following incentive 
constraints: for each player p and each of his actions j G Sp, 

E - J--. > 0- (5) 

{i,s-p)es 

We write these incentive constraints in matrix form as Cx > 0. Thus C is an |5p|) x M matrix. By definition, a 
CE is also a CCE. 

The problem of computing a maximum social welfare CCE can be formulated as the LP 

max w'^x (CP) 
Cx>0, x>0,^Xs = l. 

3 The Deviation-Adjusted Social Welfare Problem 

Consider the dual of (P), 

min t (D) 
U'^y + w<tl 

y>o. 

We label the {p,i,j)-th element of y € (corresponding to row {p,i,j) of U) as yfj. This is an LP with a 
polynomial number of variables and an exponential number of constraints. Given a separation oracle, we can solve 
it in polynomial time using the ellipsoid method. A separation oracle needs to determine whether a given {y, t) is 
feasible, and if not output a hyperplane that separates {y, t) from the feasible set. We focus on a restricted form of 
separation oracles, which outputs a violated constraint for infeasible points.'^ Such a separation oracle needs to solve 
the following problem: 

Problem 1. Given (y, t) with y > 0, determine if there exists an s such that {Us)'^y + Ws > t; if so output such an s. 
The left-hand-side expression {Us)'^y + Ws is the social welfare at s plus the term {Us)'^y- Observe that the j)-th 
entry of Us is — u^js_^ if Sp = i and is zero otherwise. Thus {UsY^V = X^jeSp j (^s ~ ^js_p) • '^^^ 
reexpress {Us)^y -|- in terms of deviation-adjusted utilities and deviation-adjusted social welfare. 

Deiuiition 3. Given a game, and a vector y e such that y > 0, the deviation-adjusted utiUty /or player p under 
pure profile s is 

<(2/) = <+E 

The deviation-adjusted social welfare is Wsiy) = J2p 

By construction, the deviation-adjusted social welfare Ws{y) = J2p'^s + 

J2pJ2jeSp Vspj ~ ^js_p) ~ {Us)^y + Ws. Therefore, Problem 1 is equivalent to the following deviation- 
adjusted social welfare problem. 

Definition 4. For a game representation, the deviation-adjusted social welfare problem is the following: given an 
instance of the representation and rational vector {y, t) € such that y > 0, determine if there exists an s such 

that the deviation-adjusted social welfare uis{y) > t; if so output such an s. 

^ This is a restriction because in general there exist separating hyperplanes other than the violated constraints. For example Pa- 
padimitriou and Roughgarden [2008] 's algorithm for computing a sample CE uses a separation oracle that outputs a convex 
combination of the constraints as a separating hyperplane. 



Proposition 1. If the deviation-adjusted social welfare problem can he solved in polynomial time for a game repre- 
sentation, then so can the problem of computing the maximum social welfare CE. 

Proof. Recall that an algorithm for Problem 1 can be used as a separation oracle for {D). Then we can apply the 
ellipsoid method using the given algorithm for the deviation-adjusted social welfare problem as a separation oracle. 
This solves (D) in polynomial time. By LP duality, the optimal objective of (D) is the social welfare of the optimal 
CE. The cutting planes generated during the elhpsoid method can then be used to compute such a CE with polynomial- 
sized support. □ 

Let us consider interpretations of the dual variables y and the deviation-adjusted social welfare of a game. The 
dual (D) can be rewritten as minj,>o max,, Ws(t/). By weak duality, for a given ?y > the maximum deviation- 
adjusted social welfare maxg (y) is an upper bound on the maximum social welfare CE. So the task of the dual (D) 
is to find y such that the resulting maximum deviation-adjusted social welfare gives the tightest bound.^ At optimum, 
y corresponds to the concept of "shadow prices" from optimization theory; that is, yfj equals the rate of change in 
the social welfare objective when the constraint {p, is relaxed infinitesimally. Compared to the maximum social 
welfare CE problem, the maximum deviation-adjusted social welfare problem replaces the incentive constraints with 
a set of additional penalties or rewards. Specifically, we can interpret y as a set of nonnegative prices, one for each 
incentive constraint {p, of (P). At strategy profile s, for each incentive constraint {p, we impose a penalty 
equal to yfj times the amount the constraint (p, ^, j) is violated by s. Note that the penalty can be negative, and is zero 
if Sp ^ i. Then Ws{y) is equal to the social welfare of the modified game. 

Practical computation. The problem of computing the expected utility (EU) given a mixed strategy profile has 
been estabUshed as an important subproblem for both the sample NASH problem and the sample CE problem, both in 
theory [Daskalakis et ai, 2006; Papadimitriou & Roughgarden, 2008] and in practice [Blum et ai, 2006; Jiang et al, 
201 1]. Our results suggest that the deviation-adjusted social welfare problem is of similar importance to the optimal 
CE problem. This connection is more than theoretical: our algorithmic approach can be turned into a practical method 
for computing optimal CE. In particular, although it makes use of the ellipsoid method, we can easily substitute a 
more practical method, such as simplex with column generation. In contrast, Papadimitriou and Roughgarden [2008]'s 
algorithmic approach for reduced forms makes two nested appUcations of the ellipsoid method, and is less likely to 
be practical. Furthermore, even for representations without a polynomial-time algorithm for the deviation-adjusted 
social welfare problem, a promising direction would be to formulate the deviation-adjusted social welfare problem as 
a integer program or constraint program and solve using e.g. CPLEX. 

3.1 The Weighted Deviation-Adjusted Social Welfare Problem 

For the max-min welfare CE problem, we can form the dual of (3), 

min t (6) 

U^y + Y.^'p^" ^ti (7) 
p 

y>0, v>0 

p 

This is again an LP with polynomial number of variables and exponential number of constraints; specifically, block (7) 
is exponential. We observe that (7) is similar to the corresponding block in (D), except for the weighted sum J2p VpU^ 
instead of the social welfare w. Thus, in order to express the left-hand side of (7) we need notions slightly different 
from those given in Definition 3, which we call weighted deviation-adjusted utility and weighted deviation-adjusted 
social welfare. 

Defuiition 5. Given a game, a vector y G such that y >0, and a vector v e R" such that v >0 and Vp = 1, 
the weighted deviation-adjusted utiUty /or p/aj^erp under pure profile s is 

vP{y,v) = VpvF, -{- ^ - 



^ An equivalent perspective is to view y as Lagrange multipliers, and the optimal deviation-adjusted SW problem as the Lagrangian 
relaxation of (P) given the multipliers y. 



The weighted deviation-adjusted social welfare wtt)s(y,w) = J^p'^siV''^)- 

Following analysis similar to that given above, the following problem serves as a separation oracle of LP (6). 

Definition 6. For a game representation, the weighted deviation-adjusted social welfare problem is the following: 
given an instance of the representation, and rational vector {y, v, t) € such that y > 0, v > and Vp 

1 , determine if there exists an s such that the deviation-adjusted social welfare Wsiy) > t; if so output such an s. 

Proposition 2. If the weighted deviation-adjusted social welfare problem can be solved in polynomial time for a game 
representation, then the problem of computing the max-min welfare CE is in polynomial time for this representation. 

It is straightforward to see that the deviation-adjusted social welfare problem reduces to the weighted deviation- 
adjusted social welfare problem. In all representations that we consider in this chapter, the weighted and unweighted 
versions have the same structure and thus the same complexity. 

3.2 The Coarse Deviation-Adjusted Social Welfare Problem 

For the optimal social welfare CCE problem, we can form the dual of (CP) 

min t (8) 
C^y -\-w <tl 

y>o 

Definition 7. We label the {p, j)-th element ofy as y^. Given a game, and a vector y € R^p such that y > 0, the 
coarse deviation-adjusted utility for player p under pure profile s is 

<(2/)=<+E?^'«-4.-J- 
The coarse deviation-adjusted social welfare is Wsiy) = u^iv)- 

Proposition 3. If the coarse deviation-adjusted social welfare problem can be solved in polynomial time for a game 
representation, then the problem of computing the maximum social welfare CCE is in polynomial time for this repre- 
sentation. 

The coarse deviation-adjusted social welfare problem reduces to the deviation-adjusted social welfare problem. To 
see this, given an input vector y for the coarse deviation-adjusted social welfare problem, we can construct an input 
vector y' e for the deviation-adjusted social welfare problem with y'^ = yj for all p € J\f and «, j G Sp. 

4 The Deviation-Adjusted Social Welfare Problem for Specific Representations 

In this section we study the deviation-adjusted social welfare problem and its variants on specific representations. 
Depending on the representation, the deviation-adjusted social welfare problem is not always solvable in polynomial 
time. Indeed, Papadimitriou and Roughgarden [2008J showed that for many representations the problem of optimal 
CE is NP-hard. Nevertheless, for such representations we can often identify tractable subclasses of games. We will 
argue that the deviation-adjusted social welfare problem is a more useful formulation for identifying tractable classes 
of games than the separation problem formulation of Papadimitriou and Roughgarden [2008], as the latter only applies 
to reduced-form-based representations. 

4.1 Reduced Forms 

Papadimitriou and Roughgarden [2008] gave the following reduced form characterization of representations. 

Definition 8 ([Papadimitriou & Roughgarden, 2008]). Consider a game G = (TV, {Sp}peAr, {m^Ipg.v)- For p = 
let Pp = {Cp ... Cp''} be a partition of S-p into rp classes. The set V = {Pi, Pn} of partitions is a 
reduced form of G if = u^, whenever (1) Sp — s'^ and (2) both s-p and s'_p belong to the same class in Pp. The 
size of a reduced form is the number of classes in the partitions plus the bits required to specify a payoff value for each 
tuple {p, k, i) where l<p<n, l<k<rp and i € Sp. 



Intuitively, the reduced form imposes tlie condition that p's utility for choosing an action Sp depends only on which 
class in the partition Pp the profile of the others' actions belongs to. 

Papadimitriou and Roughgarden [2008] showed that several compact representations such as graphical games and 
anonymous games have natural reduced forms whose sizes are (roughly) equal to the sizes of the representation. We 
say such a compact representation has a concise reduced form. Intuitively, such a reduced form describes the structure 
of the game's utility functions. 

Example 1. A graphical game [Keams et ah, 2001] is associated with a graph (A/", E), such that player p's utility de- 
pends only on her action and the actions of her neighbors in the graph. The sizes of the utility functions are exponential 
only in the degrees of the graph. Such a game has a natural reduced form where the classes in Pp are identified with 
the pure profiles of p's neighbors, i.e., s_p and s'_p belong to the same class if and only if they agree on the actions 
of p's neighbors. The size of the reduced form is exactiy the number of utility values required to specify the graphical 
game's utility functions. □ 

Let Sp{k, £) denote the set of pure strategy profiles s such that Sp = i and s-p is in the A;-th class Cp of Pp, and 
let denote the utility of p for that set of strategy profiles. Papadimitriou and Roughgarden [2008] defined the 
following Separation Problem for a reduced form. 

Definition 9 ([Papadimitriou & Rougligarden, 2008]). Let V be a reduced form for game G. The Separation Prob- 
lem/or V is the following: Given rational numbers jp{k, £) for all p G {!,..., n}, k € {1, . . . , rp}, and I € Sp, is 
there a pure strategy profile s such that J2p,k,e:seSp(k e) 7p(^> ^) < 0? If so, find such an s. 

Sinces e <Sj,(fc,£) implies Sp = £,theleft-handsideoftheaboveexpressionisequivalentto^p^j..j,g5 ^^.^ )lp{k,Sp) 
Furthermore, since s belongs to exactly one class in Pp, the expression is a sum of exactly n summands, one for each 
player. 

Papadimitriou and Roughgarden [2008] proved that if the separation problem can be solved in polynomial time, 
then a CE that maximizes a given linear objective in the players' utilities can be computed in time polynomial in the size 
of the reduced form. How does Papadimitriou and Roughgarden [2008] 's sufficient condition relate to ours, provided 
that the game has a concise reduced form? We show that the class of reduced form games for which our weighted 
deviation-adjusted social welfare problem is polynomial-time solvable contains the class for which the separation 
problem is polynonual-time solvable. 

Proposition 4. Let V be a reduced form for game G. Suppose the separation problem can be solved in polynomial 
time. Then the weighted deviation-adjusted social welfare problem can be solved in time polynomial in the size of the 
reducedform. 

Proof. First we observe that if a game G has a reduced form V, then its deviation-adjusted utilities (and weighted 

deviation-adjusted utilities) also satisfy the partition structure specified by T', i.e., given y and v, the weighted deviation- 
adjusted utihty vPg{y, v) depends only on a player's action Sp and the class in Pp that S-p belongs to. To see why, 
suppose S-p e Cp. Then 

which depends only on i and k. This proves the following, which will be useful later. 

Lemma 1. Let V be a reduced form for game G. 

L For all y G R^, v E R",/or all players p, Sp G Sp, and for all s^p, s'_p G S-p, ifs-p and s'_p are in the same 
class in Pp then the weighted deviation-adjusted utilities uf^ [y, v) = ^, (y, v). 

2. Write the weighted deviation-adjusted utility for player p, given her pure strategy £ € Sp and class G^, as 
^(/s e) ^) ('^^^^ defined by the above). We have 

jeSp 



Given an instance of the weighted deviation-adjusted social welfare problem with a game with reduced form V 
and rational vectors y G M^, v G and i G R, we construct an instance of the separation problem by letting 
7p(A:, I) = t/n — ii^j^ {y, v), where ii^^ {y, v) is as defined in Lenuna 1 and can be efficiently computed given the 
reduced form. Recall that the separation problem asks for pure profile s such that ^ e-seSp{k e) 7p(^' ^) < 0' the 
left hand side of which is a sum of n terms. By construction, for all s, £.g^g (j, jp{k,£) < if and only if 

12p Sfc se5 (k s ) (V"' ^ '^'("fe s ) ^)) ^'^'^ since the left hand side is a sum of n terms, this holds if and only if 
'wP{y, v) > t. Therefore the weighted deviation-adjusted social welfare problem instance has a solution s if and only 
if the corresponding separation problem instance has a solution s, and a polynomial-time algorithm for the separation 
problem can be used to solve the weighted deviation-adjusted social welfare problem in polynomial time. □ 

We now compare the the weighted deviation-adjusted social welfare problem with the optimal social welfare 
problem for these representations. We observe from Lemma 1 that the weighted deviation-adjusted social welfare 
problem can be formulated as an instance of the optimal social welfare problem on another game with the same 
reduced form but different payoffs. Can we claim that the existence of a polynomial-time algorithm for the optimal 
social welfare problem for a representation implies the existence of a polynomial-time algorithm for the weighted 
social welfare problem (and thus the optimal CE problem)? This is not necessarily the case, because the representation 
might impose certain structure on the utility functions that are not captured by the reduced forms, and the polynomial- 
time algorithm for the optimal social welfare problem could depend on the existence of such structure. The weighted 
deviation-adjusted social welfare problem might no longer exhibit such structure and thus might not be solvable using 
the given algorithm. 

Nevertheless, if we consider a game representation that is "completely characterized" by its reduced forms, the 
weighted deviation-adjusted social welfare problem is equivalent to the decision version of the optimal social welfare 
outcome problem for that representation. To make this more precise, we say a game representation is a reduced-form- 
based representation if there exists a mapping from instances of the representation to reduced forms such that it maps 
each instance to a concise reduced form of that instance, and if we take such a reduced form and change its payoff 
values arbitrarily, the resulting reduced form is a concise reduced form of another instance of the representation. 

Corollary 1. For a reduced-form-based representation, if there exists a polynomial-time algorithm for the optimal 
social welfare problem, then the optimal social welfare CE problem and the max-min welfare CE problem can be 
solved in polynomial time. 

Of course, this can be derived using the separation problem for reduced forms without the deviation-adjusted social 
welfare formulation. On the other hand, the deviation-adjusted social welfare formulation can be applied to represen- 
tations without concise reduced forms. In fact, we will use it to show below that the connection between the optimal 
social welfare problem and the optimal CE problem appUes to a wider classes of representations than just reduced- 
form-based representations. 

4.2 Linear Reduced Forms 

One class of representations that does not have concise reduced forms are those that represent utility functions as sums 
of other functions, such as polymatrix games and the hypergraph games of Papadimitriou and Roughgarden [2008]. In 
this section we characterize these representations using linear reduced forms, showing that linear-reduced-form-based 
representations satisfy a property similar to Corollary 1 . 

Roughly speaking, a linear reduced form has multiple partitions for each agent, rather than just one; an agent's 
overall utility is a sum over utility functions defined on each of that agent's partitions. 

Definition 10. Consider a game G = {Af, {Sp}p^j\i-, {u^jpgjv")- Forp = 1, . . . ,n, let Pp = {Pp,i, • • • , Pp,tp }. where 
Pp^q = {Cp g . . . Cpfq } is a partition of S-p into Vpq classes. The set V = {Pi, . . . , P„} is a linear reduced form of 
G if for each p there exist u^'^, . . . , u^'^p G such that for all s, vP^ = u^'"^, and for each q < tp, uj'* = 
whenever (1) Sp = s'p and (2) both S-p and s'_p belong to the same class in Pp^q. The size of a reduced form is 
the number of classes in the partitions plus the bits required to specify a number for each tuple {p, q, k, t) where 
1 < p < n, \ < q < tp, 1 < k < Tpq and i G Sp. 

We write u^j?^-^ for the value corresponding to tuple {p,q,k,£), and for k = {ki , . . . , kt^,) we write u^y^^-^ = 



Example 2 (polymatrix games). In a polymatrix game, each player's utility is the sum of utilities resulting from her 
bilateral interactions with each of the n — 1 other players: = X^p'^^p ^s],^^^ ^s^,, where A^p S j^I^pIxI'S'p'I 
and e ]Rl ''i'l is the unit vector corresponding to Sp. The utility functions of such a representation require only 
J2pp'eM \^p\ ^ \^p'\ values to specify. Polymatrix games do not have a concise reduced-form encoding, but can 
easily be written as linear-reduced-form games. Essentially, we create one partition for every matrix game that an 
agent plays, with each class differing in the action played by the other agent who participates in that matrix game, and 
containing all the strategy profiles that can be adopted by all of the other players. Formally, given a polymatrix game, 
we construct its linear reduced form with Pp = {Pp,g}qeN'\{p}' a^d Pp^g = {Cp^j^gs^ with = {s-p\sq = £}. 

a 

Most of the results in Section 4.1 straightforwardly translate to linear reduced forms. 

Lemma 2. Let V be a linear reduced form for game G. Then for all y G v G R", for all players p, there exist 

vP'^{y^ w), . . . , {t^'*''(t/, v) e R*^ such that the weighted deviation-adjusted utilities v) = vP''^{y, v), and 

for all q < tp, Sp S Sp and s-p,s'_p G S-p, if S-p and s'_p are in the same class in Pp,q, then u^'^g_^{y,v) = 

k ^* 

Write the weighted deviation-adjusted utility for player p, her pure strategy £ G Sp and classes Cp^^ , ■ ■ ■ , Cpf as 
^("k e) ^) ^fi^*"^ k = {ki, . . . ,ktj,). Furthermore, we have 

^('k,£)(y.^) = ^p«('k,£) + <j(<k,^) - "('k,,-))- 

Corollary 2. For a linear-reduced-form-hased representation, if there exists a polynomial-time algorithm for the op- 
timal social welfare problem, then the optimal social welfare CE problem and the max-min welfare CE problem can 
be solved in polynomial time. 

Graphical Polymatrix Games A polymatrix game may have graphical-game-like structure: player p's utility may 

depend only on a subset of the other player's actions. In terms of utility functions, this corresponds to A^p = for 
certain pairs of players p,p' . As with graphical games, we can construct the (undirected) graph G = (M, E) where 
there is an edge {p, p'} & Eif ^ or^^' ^ ^Q. We call such a game a graphical polymatrix game. This can also 
be understood as a graphical game where each player p's utility is the sum of bilateral interactions with her neighbors. 

A tree polymatrix game is a graphical polymatrix game whose corresponding graph is a tree. Consider the optimal 
CE problem on tree polymatrix games. Since such a game is also a tree graphical game, Papadimitriou and Rough- 
garden [2008] 's optimal CE algorithm for tree graphical games can be applied. However, this algorithm does not run 
in polynomial time, because the representation size of tree polymatrix games can be exponentially smaller than that 
of the corresponding graphical game (which grows exponentially in the degree of the graph). However, we can give a 
different polynomial-time algorithm for this problem. 

Theorem 1. Optimal CE in tree polymatrix games can be computed in polynomial time. 

Proof. It is sufficient to give an algorithm for the deviation-adjusted social welfare problem. Using an argument similar 
to that given in Example 2, tree polymatrix games have a natural linear reduced form, and it is straightforward to verify 
that tree polymatrix games are a linear-reduced-form-based representation. By Corollary 2 it is sufficient to construct 
an algorithm for the optimal social welfare problem. 

Let Np be the set of players in the subtree rooted at p. Suppose p's parent in the tree is q. Let the social welfare 
contribution of Np be the social welfare of players in Np minus A^'^eg^. Let the social welfare contribution of the 
root player be the social welfare of J\f. Then the social welfare contribution of Np depends solely on the pure strategy 
profile restricted to Np. 

The following dynamic programming algorithm solves the optimal social welfare problem in polynomial time. We 
go from the leaves to the root of the tree. Each child g of p passes to its parent the message {^^^''^'IsgSS,' where 
w^"'^" is the optimal social welfare contribution of Ng provided that q plays Sg. Given the messages from all of p's 



children gi, we can compute the message of p as follows: for each Sp G Sj 



w'""''" = max > Iw^'^'**'^ + eT AP'«-*es 



max 



w 



The second equality is due to the fact that the j-th summand depends only on Sq. . It is straightforward to verify that 
the optimal social welfare is maxg^ ^^r,sr where r is the root player, and that the algorithm runs in polynomial time. 
The corresponding optimal pure strategy profile can be constructed by going from the root to the leaves. □ 

This algorithm can be straightforwardly extended to yield a polynomial-time algorithm for optimal CE in graphical 
polymatrix games with constant treewidth, for hypergraphical games [Papadimitriou & Roughgarden, 2008] on acychc 
hypergraphs, and more generally for hypergraphs with constant hypertree-width. 



4.3 Representations with Action-Specific Structure 

The above results for reduced forms and linear reduced forms crucially depend on the fact that the partitions (i.e., the 
structure of the utility functions) depend on p but do not depend on the action chosen by player p. 

There are representations whose utility functions have action-dependent structure, including congestion games 
[Rosenthal, 1973], local effect games 

[Leyton-Brown & Tennenholtz, 2003], and action-graph games [Jiang et ai, 2011]. For such representations, we can 
define a variant of the reduced form that has action-dependent partitions. However, unlike both the reduced form 
and linear reduced form, the weighted deviation-adjusted utilities no longer satisfy the same partition structure as 
the utilities. Intuitively, the weighted deviation-adjusted utility at s has contributions from the utilities of the strategy 
profiles when player p deviates to different actions. Whereas for linear reduced forms these deviated strategy profiles 
correspond to the same class as s in the partition, we now consider different partitions for each action to which p 
deviates. As a result the weighted deviation-adjusted social welfare problem has a more complex form that the optimal 
social welfare problem. 



Singleton Congestion Games leong et al. [2005] studies a class of games called singleton congestion games and 
showed that the optimal PSNE can be computed in polynomial time. Such a game can be formulated as an instance 
of congestion games where each action contains a single resource, or an instance of symmetric AGGs where the orily 
edges are self edges. 

Formally, a singleton congestion game is specified by {M, A, {/" }aeA) where J\f = 1 , . . . , n is the set of players, 
A the set of actions, and for each action a G A, f" : [n] — > R. The game is symmetric; each player's set of actions 
Sp = A. Each strategy profile s induces an action count c{a) = \{p\sp = a}| on each a: the number of players 
playing action a. Then the utility of a player that chose a is /"(c(a)). The representation requires 0(|^|n) numbers 
to specify. 

We now show that the optimal social welfare CCE problem can be computed in polynomial time for singleton 
congestion games. Before attacking the problem, we first note that the optimal social welfare problem can be solved 
in polynomial time by a relatively straightforward dynamic-programming algorithm which is a simplified version of 
leong et al. [2005] 's algorithm for optimal PSNE in singleton congestion games. First observe that the social welfare 
of a strategy profile can be written in terms of the action counts: 

«;, = 5^c(a)r(c(a)). 

The optimal social welfare problem is equivalent to finding a vector of action counts that sums to n and maximizes the 
above expression. The social welfare can be further decomposed into contributions from each action a. The dynamic- 
programming algorithm starts with a single action and adds one action at a time until all actions are added. At each 
iteration, it maintains a set of tuples {(n', )}i<n'<n> specifying that the best social welfare contribution from the 
current set of actions is when exactly n' players chose actions in the current set. 



Consider the optimal social welfare CCE problem. Can we leverage the algorithm for the optimal social welfare 
problem to solve the coarse deviation-adjusted social welfare problem? Our task here is shghtly more comphcated: in 
general the coarse deviation-adjusted social welfare problem no longer has the same symmetric structure due to the 
fact that y can be asymmetric. However, when y is player- symmetric (that is, = yj for aU pairs of players {p, p')), 
then we recover symmetric structure. 

Lemma 3. Given a singleton congestion game and player-symmetric input y, the coarse deviation-adjusted social 

welfare problem can be solved in polynomial time. 

Proof. The coarse deviation-adjusted social welfare can be written as 

ws{y) = + E - E E y>%-. 



E 




c{a)f"{c{a)) 1 + V yn - (n - c{a))f'^{c{a) + l)yg 



The contribution from each action a depends only on c{a). Therefore, using a similar dynamic -programming algo- 
rithm as above we can solve the coarse deviation-adjusted social welfare problem in polynomial time. □ 

Therefore if we can guarantee that during a run of ellipsoid method for (8) all input queries y to the separation 
oracle are symmetric, then we can apply Lemma 3 to solve the problem in polynomial time. We observe that for any 
symmetric game, there must exist a symmetric CE that optimizes the social welfare. This is because given an optimal 
CE we can create a mixture of permuted versions of this CE, which must itself be a CE by convexity, and must also 
achieve the same social welfare by symmetry. However, this argument in itself does not guarantee that the y we obtain 
by the method above will be symmetric. Instead, we observe that if we solve (8) using a ellipsoid method with a 
player-symmetric initial ball, and use a separation oracle that returns a player-symmetric cutting plane, then the query 
points y will be player-symmetric. We are able to construct such a separation oracle using a symmetrization argument. 

Theorem 2. Given a singleton congestion game, the optimal social welfare CCE can be computed in polynomial time. 

The proof is given in Appendix A. Our approach for singleton congestion games crucially depends on the fact that the 
coarse deviation profile y^ does not care which action it is deviating from. This allowed us to (in the proof of Lemma 3) 
decompose the coarse deviation-adjusted social welfare into terms that only depend on the action count on one action. 
The same approach cannot be directly applied to solve the optimal CE problem, because then the deviation profile 
would give a different y^ for each action i that p deviates from, and the resulting expression for deviation-adjusted 
social welfare would involve suimnands that depend on the action counts on pairs of actions. 



5 Conclusion and Open Problems 

We have proposed an algorithmic approach for solving the optimal correlated equilibrium problem in succinctly rep- 
resented games, substantially extending a previous approach due to Papadimitriou and Roughgarden [2008]. In partic- 
ular, we showed that the optimal CE problem is tractable when the deviation-adjusted social welfare problem can be 
solved in polynomial time. We generalized the reduced forms of Papadimitriou and Roughgarden [2008] to show that 
if a representation can be characterized by "linear reduced forms", i.e. player-specific linear functions over partitions, 
then for that representation, the deviation-adjusted social welfare problem can be reduced to the optimal social welfare 
problem. Leveraging this result, we showed that the optimal CE problem is tractable in graphical polymatrix games 
on tree graphs. We also considered the problem of computing the optimal coarse correlated equilibrium, and derived 
a similar sufficient condition. We used this condition to prove that the optimal CCE problem is tractable for singleton 
congestion games. 

Our work points the way to a variety of open problems, which we briefly summarize here. 

Price of Anarchy. Our results imply that for compactly represented games with polynomial-time algorithms for 
the optimal social welfare problem and the weighted deviation-adjusted social welfare problem, the Price of Anarchy 
(POA) for correlated equilibria (i.e., the ratio of social welfare under the best outcome and the worst correlated equi- 
librium) can be computed in polynomial time. Similarly for the Price of Total Anarchy (i.e., the ratio of social welfare 



under the best outcome and the worst coarse correlated equiUbrium). There is an extensive literature on proving bounds 
on the POA for various solution concepts and for various classes of games. One line of research that is particularly 
relevant to our work is the "smoothness bounds" method pioneered by Roughgarden [2009]. In particular, that work 
showed that if a certain smoothness relation can be shown to hold for a class of games, then it can be used to prove 
an upper bound on POA for these games that holds for many solution concepts including pure and mixed NE, CE and 
CCE. More recently, Nadav and Roughgarden [2010] gave a primal-dual LP formulation for proving POA bounds and 
showed that finding the best smoothness coefficients corresponds to the dual of the LP for the POA for average coarse 
correlated equihbrium (ACCE), a weaker solution concept than CCE. The primal-dual LP formulation of Nadav and 
Roughgarden [2010] and our LPs (P) and (D) are equivalent up to scaling; however whereas Nadav and Roughgarden 
[2010] focused on the task of proving POA upper bounds for classes of games, here we focus on computing the opti- 
mal CE / CCE and POA for individual games. One interesting direction is to use our algorithms together with an game 
instance generator to automatically find game instances with large POA, thus improving the lower bounds on POA for 
given classes of games. 

Complexity separations. We have shown that for singleton congestion games, the optimal social welfare problem 
and the optimal CCE problem are tractable while the complexity of the optimal CE problem is unknown. An open 
problem is to prove a separation of the complexities of these problems for singleton congestion games or for another 
class. Another related problem is the optimal PSNE problem, which can be thought of as the optimal CE problem 
plus integer constraints on x. We do not know the exact relationship between the optimal PSNE problem and the other 
problems. For example the optimal PSNE problem is known to be tractable for singleton congestion games [leong 
et al., 2005] while we do not know how to solve the optimal CE problem. On the other hand for tree polymatrix games 
we showed the CE problem is in polynomial time, while the complexity of the PSNE problem is unknown. 

Necessary condition for tractability. Another open question is the following: is tractability of the deviation- 
adjusted social welfare problem a necessary condition for tractabihty of the optimal CE problem? We know (e.g., 
from Grotschel et al. [1988]) that the separation oracle problem for the dual LP (D) is equivalent to the problem 
of optimizing an arbitrary linear objective on the feasible set of (D). However this in itself is not enough to prove 
equivalence of the deviation-adjusted social welfare problem and the optimal CE problem. First of all the separation 
oracle problem is more general: it allows cutting planes other than constraints corresponding to pure strategy profiles. 
Furthermore, (D) has a particular objective, but optimizing an arbitrary linear objective means allowing the objective 
to depend on y as well as t. If we take the dual of such an LP with (e.g.) objective r'^y + t for some vector r € R^, 
we get a generalized version of the optimal CE problem, with constraints Ux > r instead of Ux > 0. 

Relaxations and approximations. Another interesting direction worth exploring is relaxations of the incentive 
constraints of these problems, either as hard bounds or as soft constraints that add penalties to the objective, as well 
as the problem of approximating the optimal CE. For these problems we can define corresponding variants of the 
deviation-adjusted social welfare problem as sufficient conditions, but it remains to be seen whether one can prove 
concrete results, e.g., for approximating optimal CE for specific representations for which the exact optimal CE prob- 
lem is hard. 

Communication complexity of uncoupled dynamics. Hart and Mansour [2010] considered a setting in which 
each player is informed only about her own utility function, and analyzed the communication complexity for so-called 
uncoupled dynamics to reach various kinds of equilibrium. They used a straightforward adaptation of Papadimitriou 
and Roughgarden [2008] 's algorithm for a sample CE to show that a CE can be reached using polynomial amount 
of communication. We can consider the question of reaching an optimal CE by uncoupled dynamics. Our approach 
can be straightforwardly adapted to this setting, reducing the problem to finding a communication protocol for the 
uncoupled version of the deviation-adjusted social welfare problem in which each player knows only her own utility 
function. 

Proposition 5. If there is a polynomial communication protocol for the uncoupled deviation-adjusted social welfare 
problem, then there is a polynomial communication protocol for the optimal CE problem. 

At a high level, the protocol has a center running the ellipsoid method on {D), using the communication protocol for 
the uncoupled deviation-adjusted social welfare problem as a separation oracle. An open problem is whether there 
exist more "natural" types of dynamics that converge to optimal CE. For example, there is extensive literature on no- 
intemal-regret learning dynamics that converges to the set of approximate CE in a polynomial number of steps. Can 
such dynamics be modified to yield optimal CE? 
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A Proof of Theorem 2 



Proof. As argued in Section 4.3, it is sufficient to construct a separation oracle for (8) that returns a player- symmetric 
cutting plane. The cutting plane corresponding to a pure strategy profile solution s of the coarse deviation-adjusted 
social welfare problem is not player- symmetric in general; but we can symmetrize it by constructing a mixture of 
permutations of s. Since by symmetry each permuted version of s correspond to a violated constraint, the resulting 
cutting plane is still correct and is symmetric. Enumerating all permutations over players would be exponential, but it 
turns out that for our purposes it is sufficient to use a small set of permutations. 

Formally, let ttj be the permutation over the set of players Af that maps each pto p + i mod n. Then the set of 
permutations {TTi}o<i<n-i corresponds to the cyclic group. 

Suppose s is a solution of the coarse deviation-adjusted social welfare problem with symmetric input y. The 
corresponding cut (violated constraint) is {Cs)'^y+Ws < t. Recall that the (p, j)-th entry of Cg is Cf'-' = (uf — ). 
For a permutation tt over M, write the permuted profile induced by tt, i.e. s'" = (s^(i) , . . . , S7r(„) ) . Then is also 
a solution of the coarse deviation-adjusted social welfare problem. Form the following convex combination of n of the 
constraints of (8): 



i . We claim that this cutting plane is 



The left-hand side can be simpUfied to Wg + {CaY^y where C's = ^ Y^=o 
player-symmetric, meaning = for all pairs of players p and all j G ^. This is because 



^ n — l -,71 — 1 

i=0 1=0 



y c{a)r{c{a)) - (n - c{j))f{c{j) + 1) 



= C 



;P ,J 



This concludes the proof. 



